System and a method for surveying graphical objects on a screen display

ABSTRACT

A method, a computer program, or a computerized system for monitoring graphical content of a screen display by the actions of receiving, by a first software program executed by a processor of a computerized device communicatively coupled to a first server via a communication network, at least one parameter characterizing a graphical object; monitoring a stream of data received by a second software program executed by the processor of the computerized device communicatively coupled to a second server via the communication network, to capture the at least one parameter characterizing a graphical object; monitoring a stream of data between the second software program and a screen display of the computerized device to capture at least one graphical object associated with the at least one parameter characterizing the graphical object; and capturing the graphical object.

FIELD

The method and apparatus disclosed herein are related to surveying userinteraction with computer generated content, and, more particularly butnot exclusively, to surveying graphical objects presented to the user ona screen display of a computerized device, and, more particularly butnot exclusively, to automatically detect, collect, and analyze graphicalobjects presented to the user on a screen display of a computerizeddevice.

BACKGROUND

Much research is done analyzing the experience a user may have whensurfing the world wide web. The raw data for the research may be knownas clickstream data or clickpath data. Various software programs such asbrowsers, as well as websites, may collect clickstream data.

In this respect, clickstream data may include all the data that isprovided to a user of a computerized device via any of the outputs ofthe device, as well as all the data that is collected from the user of acomputerized device via any of the inputs of the device. Data, orcontent, provided to the user may include visual data presented to theuser on a screen display of the computerized device. Such visual datamay include graphical objects of various types.

However, software programs collecting clickstream data concentrate oncollecting alphanumeric data in the communication stream, rather thanthe actual content as perceived by the user. There is thus a widelyrecognized need for, and it would be highly advantageous to have, amethod and a system for surveying graphical objects on a screen display,to devoid of the above limitations.

SUMMARY

According to one exemplary embodiment there is provided a method, and/ora computer program, and/or a computerized system executing the method,and/or the computer program, for monitoring graphical content of ascreen display. The method, computer program, and/or system typicallyincludes the actions, or processes, of receiving, by a first softwareprogram executed by a processor of a computerized device communicativelycoupled to a first server via a communication network, at least oneparameter characterizing a graphical object; monitoring a stream of datareceived by a second software program executed by the processor of thecomputerized device communicatively coupled to a second server via thecommunication network, to capture the at least one parametercharacterizing a graphical object; monitoring a stream of data betweenthe second software program and a screen display of the computerizeddevice to capture at least one graphical object associated with thecaptured at least one parameter characterizing the graphical object; andcapturing the graphical object.

According to another exemplary embodiment, the method, program, orsystem may additionally include an action or process for communicatingthe captured graphical object to a server via the communication network.

According to still another exemplary embodiment, the method, program, orsystem may additionally include an action or a process for collectingparameters associated with the captured graphical object the parametersincluding at least one of: temporal parameters associated with thecapturing of the captured graphical object; spatial parametersassociated with the captured graphical object; and parameters includingparameters mode of display associated with the captured graphicalobject; and an action or a process for communicating the collectedparameters to a server via the communication network.

According to still another exemplary embodiment, the method, program, orsystem may additionally include an action or a process for determiningthe least one graphical object to capture to be at least one of motiongraphics and motion image; determining at least one of number ofsamples, rate of sampling, and period of sampling for capturing thegraphical object; and repeating the capturing the graphical objectaccording to the at least one of number of samples, rate of sampling,and period of sampling for capturing the graphical object.

According to still another exemplary embodiment, the parametercharacterizing a graphical object may be associated with a part of thegraphical object, and the method, program, or system may additionallyinclude actions or processes for collecting a plurality of screen shotsof the display screen; and arranging the plurality of screen-shots tocreate a panoramic view of the display screen, where the panoramic viewmay include two or more screen-shots, and where the two or morescreen-shots includes a recognizable graphical object.

Further according to another exemplary embodiment, the recognizablegraphical object may include two or more parts of the recognizablegraphical object, where each of the parts of the recognizable graphicalobject may include one or more parameters characterizing the respectivepart of the recognizable graphical object.

Still further according to another exemplary embodiment the parametercharacterizing a graphical object may be received, and/or monitored,and/or determined, according to one or more characteristic of a contentitem, which may include the stream of data.

Yet further, according to another exemplary embodiment, the method,program, or system for monitoring graphical content of a screen displaymay include the following actions and/or processes:

Receiving, by a software program executed by a processor of acomputerized device communicatively coupled to a communication network,one or more parameters characterizing a graphical object.

Monitoring a display of the computerized device, where the processor ofthe computerized device receives at least one advertisement via thecommunication network, where the processor displays the at least oneadvertisement on a display screen of the computerized device, where themonitoring uses the software program executed by the processor of thecomputerized device.

Recognizing the one or more graphical objects being displayed on thedisplay screen. And collecting the graphical object as displayed on thedisplay screen, and at least one parameter associated with the displayscreen, and/or another content displayed on the display screensubstantially concurrently.

Unless otherwise defined, all technical and scientific terms used hereinhave the same meaning as commonly understood by one of ordinary skill inthe relevant art. The materials, methods, and examples provided hereinare illustrative only and not intended to be limiting. Except to theextent necessary or inherent in the processes themselves, no particularorder to steps or stages of methods and processes described in thisdisclosure, including the figures, is intended or implied. In many casesthe order of process steps may vary without changing the purpose oreffect of the methods described.

Implementation of the method and system of embodiments disclosed belowinvolves performing or completing certain selected tasks or stepsmanually, automatically, or any combination thereof. Moreover, accordingto actual instrumentation and equipment of preferred embodiments of themethod and system presented herein several selected steps could beimplemented by hardware or by software on any operating system of anyfirmware or any combination thereof. For example, as hardware, selectedsteps could be implemented as a chip or a circuit. As software, selectedsteps could be implemented as a plurality of software instructions beingexecuted by a computer using any suitable operating system. In any case,selected steps of the method and system disclosed herein could bedescribed as being performed by a data processor, such as a computingplatform for executing a plurality of instructions.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments are described herein, by way of example only, withreference to the accompanying drawings. With specific reference now tothe drawings in detail, it is stressed that the particulars shown are byway of example and for purposes of illustrative discussion of thepreferred embodiments only, and are presented in order to provide whatis believed to be the most useful and readily understood description ofthe principles and conceptual aspects of the embodiment. In this regard,no attempt is made to show structural details of the embodiments in moredetail than is necessary for a fundamental understanding of the subjectmatter, the description taken with the drawings making apparent to thoseskilled in the art how the several forms and structures may be embodiedin practice.

In the drawings:

FIG. 1 is a simplified illustration of is a simplified illustration of avisual object displayed by a computational device and surveyed by asurveying system;

FIG. 2 is a simplified block diagram of a computing device such as usedfor the computational device and/or the surveying system;

FIG. 3 is a block diagram of surveying software program that may beexecuted by a terminal computational device, such as a mobile device,such as a smartphone;

FIG. 4 is a block diagram of surveying software program that may beexecuted by one or more surveying servers; and

FIG. 5 is a simplified block diagram of a general process for surveyinggraphical objects on a screen display.

DESCRIPTION OF EMBODIMENTS

The principles and operation of a method and a system for surveyinggraphical objects on a screen display according to some embodiments maybe better understood with reference to the drawings and accompanyingdescription.

Before explaining at least one embodiment in detail, it is to beunderstood that the embodiments are not limited in its application tothe details of construction and the arrangement of the components setforth in the following description or illustrated in the drawings. Otherembodiments may be practiced or carried out in various ways. Also, it isto be understood that the phraseology and terminology employed herein isfor the purpose of description and should not be regarded as limiting.

In this document, an element of a drawing that is not described withinthe scope of the drawing and is labeled with a numeral that has beendescribed in a previous drawing has the same use and description as inthe previous drawings. Similarly, an element that is identified in thetext by a numeral that does not appear in the drawing described by thetext, has the same use and description as in the previous drawings whereit was described.

The drawings in this document may not be to any scale. Differentdrawings may use different scales and different scales can be used evenwithin the same drawing, for example different scales for differentviews of the same object or different scales for the two adjacentobjects.

The present embodiments comprise a method and a system for surveyinggraphical objects displayed on a screen display of a computationaldevice. The term ‘survey’ may refer to actions such as search, detect,identify, copy, collect, crop, store, communicate, and analyze any typeof graphical objects and/or images being displayed on a screen displayof a computational device.

In this document the term ‘computing device’ may refer to any type ofcomputing machine, including but not limited to, a computer, a portablecomputer (e.g. a laptop computer), a mobile communication device, anetwork server, etc., and any combination thereof. The term ‘mobilecommunication device” may refer to devices such as a tablet, a mobiletelephone, a smartphone, etc. Such computing device or computing machinemay include any type or combination of devices, including, but notlimited to, a processing device, a memory device, a storage device,and/or a communication device. The term ‘mobile computing device’ or‘mobile device’ may refer to any type of mobile or portable computingmachine such as a laptop computer, a tablet, a mobile telephone, andparticularly a smartphone.

The term ‘communication network’ or ‘network’ may refer to any type ortechnology for digital communication including, but not limited to, theInternet, WAN, LAN, MAN, PSDN, wireless networking technology such asPLMN, cellular network, WiMAX, WLAN (Wi-Fi), WPAN (Bluetooth), etc. Theterm ‘communication network’ or ‘network’ may refer to any combinationof communication technology, and to any combination of physicalnetworks. The term ‘communication network’ or ‘network’ may refer to anynumber of interconnected communication networks that may be operated byone or many network operators.

The term ‘application’ may refer to a software program running on, orexecuted by, one or more processors of a computing devices, andparticularly by a mobile computing device such as a mobile telephone, atablet, a smartphone, etc., as well as any other mobile or portablecomputing facility. The term ‘mobile application’ may refer to anapplication executed by a mobile computing device.

The terms ‘clickstream’, ‘clickpath’, ‘clickstream data’, ‘clickpathdata’, may include all the data that is provided to a user of acomputerized device via any of the outputs of the device, as well as allthe data that is collected from the user of a computerized device viaany of the inputs of the device.

The terms ‘visual’, ‘visual object’, ‘graphical object’, and ‘mage’ mayrefer to any type of output data presented to the user on a screendisplay of the computerized device. Such visual data may include text,logo, symbols of any kind and shape, as well as images, such aspictures, video streams, etc.

The terms ‘type’, ‘class’ and ‘characteristic’ may refer to any kind ofparameter, or a parameter value, or a range of values of a particularparameter, that may be common to a plurality of ‘visuals’, ‘visualobjects’, ‘graphical objects’, etc.

The term ‘advertising’ or ‘advertisement’ may refer to content providedby a third party to a user of an application. An advertisement may beprovided to the user in any sensible form such as sound, text, image,video, etc., and combinations thereof. In this sense, for example, theadvertisement is provided to the user by the application within theapplication’s screen display, or soundtrack, or any similar medium usedby the application.

The term ‘advertisement placeholder’ (or ad placeholder) may refer to aplace within an application, and particularly to an area within a screendisplay of an application, that may receive an advertisement, or inwhich an advertisement may be placed. An ‘advertisement placeholder’ maybe associated with, or include, a bidding code.

The term ‘bidding code’ may refer to a software program that may beassociated, for example, with an advertisement placeholder, and maycommunicate with an advertisement agent to enable advertisers to bid forthe advertising place and communicate an advertisement to be displayedto the user.

One of the purposes of various embodiments is to survey, collect,correlate, index, and provide information associated with visual objectsdisplayed to a user of a computerized device. The information mayinclude any type of visual displayed to the user. The embodiments mayalso provide statistics and correlations associated with the informationsurveyed and collected. The embodiments may also provide a method and asystem for executing such survey and providing the necessaryinformation.

It is appreciated that the survey may include a large number of usersusing various computational devices, operating a large variety ofapplications, including mobile applications, visiting a large variety ofwebsites, and receiving a large variety of advertisements.

A goal of the survey is to study and analyze how a particular graphicalobject may appear on the screen in various different situations,depending on the particular user, user location, the type of thecomputational device, the type of the screen display, the websitevisited, the application used, etc.

While the survey involves users using their respective computerizeddevices, as well as various software application executed by therespective computerized devices, the user using the computerized devicesor software application is not required to perform any operation of thesurvey itself. The survey is executed by the respective computerizeddevices in the background without the user being requested to make anyentries or decisions as part of the survey operations.

It should be appreciated that the present invention, and the variousembodiments of the present invention, is a pioneering invention that canbe applied in a wide variety of settings to accomplish a wide range offunctionality. As such, the embodiments and the examples provide hereinare presented to allow the reader to gain an overall understanding ofthe invention and embodiments thereof and should not be construed aslimitations of the invention. For instance, one objective of the variousembodiments is to identify, estimate, verify, etc. presentments orexposures of a particular advertisement. Further, embodiments may alsoinclude the aspect of analyzing not just how many, and the demographicsof where the advertisements are exposed, but also how the advertisementappears on the display and the visual environment of the advertisement(e.g., other content and advertisements displayed to the user within theimmediate session). The clickstream data may beneficially indicate thatthe advertisement is presented on a screen, but alone, it cannotidentify how the advertisement is displayed and what the visualenvironment of the advertisement is (i.e., what was actually displayedto the user).

It should be appreciated that in some situations, items may be presentedon a display screen that is small, or not sufficiently large enough topresent all of the visual objects of something to be displayed, such asan advertisement. As such, the screen may only display parts or portionsof an advertisement, or object that is to be displayed. In suchsituations, the ability to recognize the displayed image is much moredifficult. In some cases, to identify the object that is intended to bedisplayed requires assembling a puzzle (i.e., of a panoramic view ofmultiple pieces of the object). Further, in some cases such activity mayrequire remote image processing. Puzzle-processing requires collectingdetailed image parameters, as it is not known what part of the image maycaptured in any particular screen-shot, and how to relate image parts toone another.

It is further important to understand that in some circumstance, objectspresented on a display may have some level of motion or animationassociated with the object. For instance, in an advertisement, aparticular object may be animated to help draw attention. As such,displayed objects may have motion (e.g., image (e.g., video) or graphics(e.g., vector graphics)). In such situations, the afore-describedpuzzle-processing may thus also have a temporal aspect (not justspatial). Embodiments of the present invention advantageously allow theidentify of how much and what part of the objects have been actuallydisplayed and viewed by the user. To answer this question, it may benecessary to determine in advance the rate of capturing screen shots,which is obviously image/advertisement dependent as one (or more) of thedetailed image parameters.

It should also be appreciated that in some cases, there may be imagesactually captured that may not be recognized (or that have been failedto be recognize) in the clickstream data, but that were recognized inthe stream of screen-shots (due to the detailed image parameters).

Advantageously, in various embodiments of the present invention, theknowledge gathered by each client processing software (e.g, detailedimage parameters) can be distributed to other client processing systems.Such knowledge may include and associate parameters to be identified inthe clickstream data and the imaging details to be recognized in thecaptured screen-shots by the image recognition software.

As such, one process is to analyze the clickstream data to know what tolook for, and then to analyze the “stream of data between the secondsoftware program and a screen display” to know when and how (rate) tocapture what, then capturing a (sequence of) screen-shot(s) of thedisplay, the captured image(s) are analyzed to determine how the objectsor advertisement(s) is/are actually displayed and its/their visualenvironment (which may require puzzle-processing), and then theembodiments may present/store/distribute the analysis results.

It should be appreciated that while the various embodiments aredescribed within the context of analyzing and tracking advertisements,the various embodiments and variations thereof can be utilized in manyenvironments in which it is desirous for a machine to be able toidentify, monitory, detect, ascertain and otherwise interpret what isdisplayed on a screen, including text as well as objects/graphics/videoetc. As a non-limiting example, embodiments could be established toassess what is presented on a display for a reCAPTCHA authentication.Embodiments of the invention may be utilized identify what is displayedand then to generate a response to the “I am not a robot” challenge.Further, embodiments of the invention may be utilized to assist theblind in identifying what may be presented on a display screen.Embodiments of the invention may also be used to monitor the displayoutput of control systems to identify if and when a malfunction takesplace. The reader will appreciate that these and many otherimplementations of the various aspects of the present invention can beutilized and the present invention is not limited to those applications,functions and embodiments presented herein.

Reference is now made to FIG. 1 , which is a simplified illustration ofa visual object 10 displayed by a computational device 11 and surveyedby a surveying system 12, according to one embodiment.

FIG. 1 shows a first user 13 using computational device 11. Thecomputational device 11 may be communicatively coupled to acommunication network 14. The computational device 11 may becommunicatively coupled via communication network 14 to a variety ofnetwork servers 15.

FIG. 1 shows computational device 11, for example, as a mobile telephoneor a smartphone. However, computational device 11 may be any type ofcomputer, including a tablet, a laptop computer, a desktop computer,etc.

Computational device 11 may include a user interface for interactingwith user 13. Such user interface may include various input and outputdevices for presenting content to user 13, and/or for receiving contentfrom user 13, or under the control of user 13. Output device may includevarious types of devices for presenting visual content to user 13, suchas a screen display 16.

Computational device 11 may also include a processor (not shown in FIG.1 ) that may execute one or more software programs 17, which may includean operating system 18, and one or more applications 19 typicallyrunning over operating system 18. Application 19 may present content 20to user 13 of computational device 11, and particularly display visualcontent on the screen display 16 of computational device 11. Such visualcontent may include one or more visual objects 10.

It is appreciated that content 20 may include any type and combinationof sensory content such as image, sound, text, etc., presented to user13. The term ‘present’ may refer to any means of providing content to ahuman including, but not limited to a screen display, a speaker, anearphone, etc. Content 20 may include content selected by user 13 aswell as content 20 selected by other computational devices, such asadvertisements.

Computational device 11 may also include a client surveying softwareprogram 21, which may be part of the surveying system 12. In otherwords, surveying system 12 may include surveying software program beinginstalled in computational device 11 and executed by a processor ofcomputational device 11.

As shown in FIG. 1 , survey system 12 may also include one or moresurveying servers 22 communicatively coupled to communication network14. Surveying server 22 may include a processor (not shown in FIG. 1 )that may execute a server surveying software program (e.g., serversoftware) 23.

As shown in FIG. 1 , survey system 12 may detect and collect one or morevisual objects 10. For example surveying software 21 may detect one ormore visual objects 10, and communicate to server software 23, forexample via communications 24, the one or more visual objects 10 and/orone or more parameters associated with the one or more visual objects10. Server software 23 may then collect, index, store, and analyze theone or more visual objects 10 and/or one or more parameters associatedwith the one or more visual objects 10.

A user 25 may then obtain the results of the survey, for example via auser interface, for example by using a terminal 26 or a similarcomputational device which may be communicatively coupled to the one ormore surveying servers 22.

Reference is now made to FIG. 2 , which is a simplified block diagram ofcomputing device 27, according to one embodiment.

As an option, the block diagram of computing device 27 of FIG. 2 may beviewed in the context of the details of the previous Figures. Of course,however, the block diagram of computing device 27 of FIG. 2 may beviewed in the context of any desired environment. Further, theaforementioned definitions may equally apply to the description below.

It is appreciated that computational device 11, and/or network server15, may be embodied as one or more computing devices 27. Computingdevices 27 can therefore represent a smartphone, a tablet, a laptopcomputer, a desktop computer, a network server 15, or a part of anetwork server 15, or a plurality of network servers 15, or adata-center, including cloud-based computing infrastructure.

In this respect, survey system 12 may be embodied using one or morecomputing devices 27. Computing device 27 may therefore be operative toprovide computing infrastructure and resources for any type and/orinstance of software component executed within survey system 12. In thisregard computing device 27 is operative to process any of the prrocessesdescribed herein.

As shown in FIG. 2 , computing device 27 may include the followingcomponents:

At least one processor unit 28 operative to execute one or more softwareprograms as disclosed with reference to FIG. 1 .

One or more memory units 29, e.g. random-access memory (RAM), anon-volatile memory such as a Flash memory, etc., operative to storesoftware programs, parameters, data, databases, digital content, etc.

One or more storage units 30 including, for example, a hard disk driveand/or a removable storage drive, representing a floppy disk drive, amagnetic tape drive, a compact disk drive, Flash drive, etc. Theremovable storage drive reads from and/or writes to a removable storageunit in a well-known manner. The storage units being operative to storesoftware programs, parameters, data, databases, digital content, etc.

One or more communication units 31 operative to communicatively couplecomputing device 27 to at least one communication network 14 asdisclosed herein and in FIG. 1 .

One or more graphic processors 32 and displays 33 operative to enable auser to operate and control computing device 27 and software programsexecuted by computing device 27.

A power supply 34 and power source 35, such as a battery, for example,if computing device 27 is a mobile, and/or portable, computing device.

One or more communication buses 36 connecting the above units.

One or more computer programs 37, or computer control logic algorithms,which may be stored in any of the memory units 29 and/or storage units30. Such computer programs, when executed, enable computing device 27 toperform various functions (set forth herein, for example). Memory units29 and/or storage units 30 and/or any other storage are possibleexamples of tangible computer-readable media.

It is appreciated that any software program of FIG. 1 may be embodied asa computer program 37 of FIG. 2 . For example, operating systems,applications 19, software program 17, operating system 18, applications19, server software 23, and/or surveying software program 21.

Returning to FIG. 1 , computational device 11 may use operating system18, which may provide applications 19 with programming tools to displaycontent on the screen display. Such tools may be known as system calls,and/or application programming interface (API), etc. Operating system 18may also provide similar tools to monitor content displayed on thescreen display. Surveying software program 21 may use such system calls,and/or API, of operating system 18 to retrieve content displayed on thescreen display, including graphical objects. These tools (OS-API) mayprovide content displayed on the screen display as computer-readabledata.

Alternatively, and/or additionally, some applications 19, such asbrowser applications, may also provide such with programming tools,e.g., API, for other programs, to monitor content that the particularapplication 19 displays on the screen display. Surveying softwareprogram 21 may use such API, of an application 19, to retrieve contentdisplayed on the screen display, including graphical objects. Thesetools (Application-API) may provide content displayed on the screendisplay as computer-readable data.

Alternatively, and/or additionally, operating system 18 may also providea tool, or a system call, or an API, to obtain a screen-shot of thescreen display, for example, in the form of ‘print-screen’. Suchscreen-shot tool may provide an instantaneous image of the entirescreen. The image may not be computer-readable data and may requiretools such as optical character recognition (OCR), object recognition,etc.

Image analysis and recognition software tools such as optical characterrecognition (OCR), object recognition, etc., may require relativelyhigher processing power to analyze a screen-shot than software toolsthat analyze machine-readable data. Therefore, surveying system 12 maydivide the processing between the surveying software 21 executed bycomputational device 11, and server software 23 executed by a server 15,according to the available resources.

Surveying system 12 may consider resources such as available processingpower (for example, of processor unit 28 of FIG. 2 ), available memoryor storage (for example, of memory 29 or storage 30 of FIG. 2 ),available electric power, for example from battery 35 of FIG. 2 ,communication bandwidth (for example, of communication unit 31 of FIG. 2).

It is appreciated that executing image analysis and recognition softwaretools such as optical character recognition (OCR), object recognition,etc., may consume electric power and drain the battery of a mobiledevice. However, communicating large amount of data, such as frequentscreen-shots, may also consume battery power and load the communicationbandwidth.

Hence, surveying system 12 may consider the current availability of thevarious resources and determine the recognition technology to be used,as well as where each recognition technology should be executed, namely,surveying software 21 executed by computational device 11, and serversoftware 23 executed by a server 15.

Reference is now made to FIG. 3 , which is a block diagram of surveyingsoftware program 21, according to one exemplary embodiment.

As an option, the block diagram of FIG. 3 may be viewed in the contextof the details of the previous Figures. Of course, however, the blockdiagram of FIG. 3 may be viewed in the context of any desiredenvironment. Further, the aforementioned definitions may equally applyto the description below.

As shown in FIG. 1 , surveying software program 21 may be installed on aterminal computational device 11, which may be a mobile device such as asmartphone. As shown in FIG. 3 , surveying software program 21 mayinclude a main routine 38, and a subroutine 39.

Main routine 38 may start with action 40 by receiving a list 41 of oneor more graphical objects to be recognized, collected, saved, andcommunicated (e.g., to server software 23). Such objects may include,for example, a visual object 10 as shown and described with reference toFIG. 1 .

List 41 may include whole graphical objects, one or more parts of one ormore graphical objects, and parameters of the graphical objects. Action40 may obtain list 41, for example, from a server 15, such as asurveying server 22 and/or server software 23.

Object parameters, as may be provided in list 41, may indicate a shapeof the object, a size of the objects, a color of the object, one or moreletters associated with the object, a universal resource locator (URL),etc. The term URL may refer to any internet address, or link (to a webaddress or to a local document, bookmark, etc.), or identifier of anytype of resource or content.

For example, action 40 may obtain list 41 upon accessing a particularcontent item such as a particular website and/or a particular webpage,etc. Such list 41 may be associated with the particular content item,and/or the type, and/or subject, and/or category, and/or characteristic,of the particular content item.

For example, action 40 may analyze the content item and may determineone or more characteristics of the content item. For example, action 40may associate the content item with a particular market, and/orindustry, and/or a product type, and/or entertainment activity, and/ orsocial environment, etc. Each such subject, and/or type, and/orcharacteristic, may be associated with a particular list 41.

Hence, upon accessing a particular content item action 40 may determinethe type, and/or subject, and/or characteristic of the content item andcommunicate to surveying server 22 and/or server software 23 a requestfor a list 41 that is associated with the particular type, and/orsubject, and/or characteristic of the currently accessed content item.

It is appreciated that main routine 38 may obtain, store, and maintainone or more lists 41. For example, as main routine 38 proceeds from onewebpage to another, main routine 38 may obtain (via action 40) andmaintain a plurality of lists 41. For example, main routine 38 may firstreceive a list 41 including general and/or more common graphical objectsthat are associated with a broader range of the particular type, and/orsubject, and/or characteristic of the currently accessed content item.Thereafter, as main routine 38 proceeds to deeper webpages, main routine38 may receive lists 41 including graphical objects that are associatedwith a narrower type, and/or subject, and/or characteristic of the newlyaccessed content item.

Main routine 38 may then proceed to action 42 to determine processingdivision for various graphical objects of list 41, as may be needed. Theprocessing division may determine what should be done by surveyingsoftware program 21, and what should be done by surveying server 22and/or server software 23. The processing division may refer to actionssuch as sampling the graphical object, analyzing the graphical object,identifying the graphical object, etc.

For example, list 41 may indicate which graphical object should beprocessed and/or recognized by the client surveying software 21, and/orwhich graphical object should be processed and/or recognized by theserver software 23.

For example, when main routine 38 maintains a plurality of lists 41,surveying software program 21 and/or server software 23 may indicatewhich list 41 should be processed and/or recognized by the clientsurveying software 21, and/or which list 41 should be processed and/orrecognized by the server software 23. Main routine 38 may then proceedto action 43 to determine sampling conditions for various graphicalobjects of list 41, as may be needed. Sampling condition may determine aparticular type of data, and/or a particular value of data, that mayform a display event that may trigger sampling of a correspondinggraphical object.

For example, sampling condition may determine if a screen shot isrequired, and/or how many screen shots are required, and/or the rate atwhich screen shots should be sampled, etc.

In this respect, main routine 38 may obtain a screenshot of the screenand then subroutine 39 may crop the image in the backend server usingimage recognition. Therefore, main routine 38 may determine in real timewhich images should be captured and which should not. Main routine 38may then determine in real time which of the captured images should becropped by main routine 38 and which of the captured images should becropped by subroutine 39 and/or the server. Main routine 38 may alsodetermine in real time the metadata (parameters) for each screenshotand/or object and/or image to determine if it is relevant (for exampleaccording to list 41). Main routine 38 may then communicate tosubroutine 39 and/or the server such metadata.

Main routine 38 may then proceed to action 44 to collect input data 45.Input data 45 may include the stream of data exchanged with any internetentity such as a website or a web page, clickstream data, clickpathdata, bidding code information, advertisement, etc.

Input data 45 may also include page source data, which may be obtainedby action 44, for example, via an application programming interface(API) of a browser or a similar software program executed bycomputational device 11, such as application 19 of FIG. 1 .

Input data 45 may also include content such as content 20 presented touser 13 by application 19. For example, some of content 20 may begenerated internally by application 19, rather than received from anexternal source. Such displayable content 20 may be obtained by action44, for example, via an application programming interface (API) ofoperating system 18 of computational device 11 of FIG. 1 .

Main routine 38 may then proceed to action 46 to analyze input 45 so asto detect one or more display events, which may require sampling of dataassociated with graphical and/or visual objects displayed to user 13 onscreen display 16 of computational device 11 of FIG. 1 . A display eventmay be determined, for example, based on one or more sampling conditionsdetermined in action 43, which may be based on parameters receive byaction 40 via list 41. Main routine 38 may then proceed to action 47 tocollect display content, as determined in action 46 to form sampledcontent 48.

A display event may be determined by analyzing content and/or dataincluded in input data 45. For example, such content and/or data mayinclude image caption text, and/or iframe, and/or JavaScript container,that may include the size and/or description of the associated object.Such content and/or data may also include an image URL, which action 46may read and analyze to determine of the text includes a description oridentification of the object. Action 46 may also read and analyze theHTML content included in input data 45. Such data as the content of theURL text, caption text, iframe, JavaScript, and/or HTML data may becompared to data of the list 41 to determine if the associated object isof interest and should be processed as detail herein.

Main routine 38 may then proceed to action 49 to save and processsampled content 48 and/or to communicate sampled content 48 or any partthereof to one or more surveying servers 22, as may have been determinedby action 42. Actions 44, 46, 47, and 49 may be repeated, or executedcontinuously, for example, as long as actions 44 may obtain new inputdata 45.

Subroutine 39 may represent an exemplary embodiment of action 47 of mainroutine 38. Namely, action 47 is implemented in further details bysubroutine 39. Subroutine 39 may start with action 50 by analyzing inputdata 45 to recognize data representing a graphical object according tothe listed objects of list 41 and their associated parameters.

As described above, the analysis executed by action 50 may be subject tothe particular list 41 obtained for the content item currently accessed.If main routine 38 maintains a plurality of lists 41, action 50 maystart with the most recent list 41. If no object is recognized accordingto the current list 41, then action 50 may use a previously obtainedlist 41. For example, action 50 may use lists 41 in reverse order ofreceiving lists 41. Other algorithms for selecting a list 41 from theplurality of lists 41 are contemplated.

It is appreciated that operating system 18, and/or application 19, maymodify the displayed content 20 to fit to the parameters of the screendisplay. For example, considering the size of the screen display, thesize of a window within the screen display, the number of pixelsavailable for display, the aspect ratio and/or orientation of the screendisplay, minimum character size, etc. Therefore, the actual relationbetween various components of the displayed content 20 may differ fromone screen display to another, or according to the instant situation.Such the actual relation between various components of the displayedcontent 20 may refer to differences in relative size and/or differencesin relative location. For example, an image may be partially displayed.

Subroutine 39 may then proceed to action 51 to determine displayparameters 52 of the one or more objects detected in action 50. Displayparameters 52 may represent when, where, and how the particular objectis displayed to user 13 on screen display 16 of computational device 11of FIG. 1 . Display parameters 52 may then be included in sampledcontent 48 communicated to one or more surveying servers 22.

It is appreciated that action 47, and/or subroutine 39, may obtain, orsample, or collect, or crop, an image of the displayed object. The terms‘obtain’, ‘sample’, ‘collect” and ‘crop’, may be used hereininterchangeably, to denote how action 47, and/or subroutine 39, mayobtain an image of an object as the object is displayed to a user of thedisplay.

It is appreciated that action 47, and/or subroutine 39, may have toobtain a complete image of the displayed object. Hence, action 47,and/or subroutine 39, may know in advance at least some property, orfeature, of the sampled object being displayed, where this property, orfeature, may be used to verify that the displayed object is sampled, orcropped, completely.

For example, such image property, or feature, that verifiescompleteness, may by a graphic property associated with chromaticity,line effect, edge effect, etc., as well as combinations thereof. Forexample, the property, or feature, of completeness may be associatedwith color, such as monochrome color, such as white or black that maysurround the image and may be used to verify completeness of sampling(cropping).

For example, the property, or feature, of completeness may be associatedwith a line, or edge, or contour, such as square, or round, or oval,etc., that may surround the image and may be used to verify completenessof sampling (cropping).

Such property, or feature, that may verify completeness of sampling, maybe provided as a display parameter provided as part of list 41. Suchproperty, or feature, that was used to verify the completeness of thesampling may then be provided as a display parameter 52. It is thereforeappreciated that to obtain a complete image of a particular object, morethan one screenshot may be required. Therefore, action 47, and/orsubroutine 39, may determine, for example based on data provided as partof list 41, how to execute a plurality of screenshots to obtain acomplete image of a particular object.

To obtain such plurality of screenshots and to obtain a complete imageof a particular object, action 47, and/or subroutine 39, may, forexample, monitor the activity of the user and obtain screenshots asfurther parts of the required image are being displayed.

For example, regarding when the particular object is displayed, action51 may determine the time in which the display of the particular objecthas started, and the length of time the particular object has beendisplayed. Such temporal data may be included as object parameters indisplay parameters 52.

For example, regarding where the particular object is displayed, action51 may determine the place on the screen display where the particularobject has been displayed. For example, in terms of pixels from theupper left corner of the screen display. For example, action 51 maydetermine the size of the particular object on the display. For example,in terms of area, for example as a number of pixels, or as a length of adiagonal between the upper left corner and the bottom right corner ofthe displayed object. Such spatial data may be included as objectparameters in display parameters 52.

For example, regarding how the particular object is displayed, action 51may determine the mode in which the particular object appears on thedisplay. For example, in terms inclusion, and/or pop-up (window orframe), hover over any other displayed content, etc. For example, action51 may determine that only part of the object is being displayed. Suchdata may be included as object parameters in display parameters 52 asdisplay mode data.

For example, action 51 may also include in display parameters 52 thedata used to initiate the sampling of the particular object, such asdata from list 41 that defined the particular objects, and/or data frominput data 45 that may have triggered the display event (see action 46)that has triggered the capture of the image of the particular object,etc.

Subroutine 39 may then proceed to action 53 to determine if ascreen-shot is required. If a screen-shot is not required, subroutine 39may proceed to action 54 to save the sampled data of the visual objectand the associated parameters as may have been determined from list 41,and/or collected from input data 45, and/or analyzed with reference todisplay parameters 52.

If a screen-shot is required, subroutine 39 may proceed to action 55 todetermine if the sampled object is a motion graphic, and/or animatedgraphic, and/or video clip, or any other type of visual data that may bechanging with time. If the sampled object is a motion graphic,subroutine 39 may proceed to action 56 to determine the rate at whichthe screen shots should be obtained and the number of screenshots to besampled, and/or the time period for sampling the motion graphic object.

Subroutine 39 may then proceed to action 57 to take the screenshots ofscreen display 16 of computational device 11 of FIG. 1 as determined inaction 56, and then to action 54 to save the screen shots with theirassociated parameters as described above (including display parameters52). The screen shots and their associated parameters may then beincluded in sampled content 48 communicated to one or more surveyingservers 22.

It is appreciated that in some cases the area of the visual contentprovided for display is larger than the area of the particular screendisplay of the particular computational device 11. In such case not allthe visual content is displayed to the user at once. For example,computational device 11 may enable the user to roll, and/or pan, and/ toshrink the visual content, and/or otherwise manually scan the visualcontent on the screen display.

In such situations, where, for example, there is no exact match betweenthe size and/or or area, and/or aspect ratio, and/or any spatialdimension of the visual content and the size, and/or area, and/or aspectratio, and/or any spatial dimension of the screen display, the actuallocation, and/or position, and or size, of the particular graphicalobject, as actually displayed on the screen display, may vary, forexample, based on the user activity.

It is appreciated that the term ‘aspect ratio’ may also refer to theorientation of the screen display, such as the positioning of the screendisplay in profile (vertically) or landscape (horizontally) positions.

Hence, in such situations, where, for example, there is no exact matchbetween the size and/or or area, and/or aspect ratio, and/or any spatialdimension of the visual content and the size, and/or area, and/or aspectratio, and/or any spatial dimension of the screen display, there may bepossible differences between the parameters of the particular visualobjects as received via list 41, the parameters of the particular visualobjects as received via input 45, and the parameters of the particularvisual objects as sampled and provided as display parameters 52.

Hence, in such situations, where, for example, there is no exact matchbetween the size and/or or area, and/or aspect ratio, and/or any spatialdimension of the visual content and the size, and/or area, and/or aspectratio, and/or any spatial dimension of the screen display, a visualand/or graphical object may be partially displayed, and/or partiallyviewed. For example, only a part of the visual and/or graphical objectmay be sampled in a screenshot.

Reference is now made to FIG. 4 , which is a block diagram of surveyingsoftware program 23 that may be executed by one or more surveyingservers 22, according to one exemplary embodiment.

As an option, the block diagram of FIG. 4 may be viewed in the contextof the details of the previous Figures. Of course, however, the blockdiagram of FIG. 4 may be viewed in the context of any desiredenvironment. Further, the aforementioned definitions may equally applyto the description below.

As shown in FIG. 4 , surveying software program 23 may start with action58 by communicating lists 41 to one or more client surveying software 21being executed on respective one or more computational devices 11. Inparallel to action 58, surveying software program 23 may execute action59, receiving sampled content 48 from various client surveying software21. As described above, sampled content 48 may include displayparameters 52.

It is appreciated that the same list 41 may be communicated to anynumber of client surveying software 21, and/or any number of differentlists 41 may be communicated to any number of client surveying software21.

Surveying software program 23 may then proceed to action 60 to processsampled content 48. For example, sampled content 48 may be scanned torecognize an object not recognized by client surveying software 21, forexample, according to processing division instructions included, forexample, in the associated list 41. If an object is identified (action61), surveying software program 23 may proceed to action 62 to determinethe object’s parameters.

If an object is not identified, surveying software program 23 mayproceed to action 63 to scan the screenshot (e.g., the image obtainedand provided in sampled content 48) to recognize a part of an object(partial object).

If an object is identified by its part (action 64), surveying softwareprogram 23 may scan further screenshots (action 65) to recognize otherparts of the same object, until all the parts of the object are scannedand identified and the object image is completed (action 66). Thereaftersurveying software program 23 may proceed to action 67 to arrange thescreenshots to form the object and to action 62 to determine theobject’s parameters. If an object is not identified by its part (action64) then surveying software program 23 may continue to search foranother object (action 60).

Surveying software program 23 may then proceed to action 68 to cropfurther objects, such as objects not listed in the current list 41, oreven objects not listed in any list 41, and to action 69 to collect dataassociated with the objects of action 68.

Surveying software program 23 may then proceed to action 70 to analyzethe accumulated content and to determine new objects and/or updatedobject parameters, and to action 71 to update the sampling conditionsand the processing divisions. Such data (objects, object parameters,processing division, sampling conditions, etc.) may be collected andmaintained in an objects database 72. Objects database 72 may then beused by action 58 to create one or more list 41, for example, based onthe identification of the content item accessed by the user.

Action 70 may also maintain an occurrence database 73 includingassociations of occurrences of objects (of objects database 72), forexample, representing threads of objects occurring in the same sessionfor a particular user.

Reference is now made to FIG. 5 , which is a simplified block diagram ofa general process 74 for surveying graphical objects on a screendisplay, according to one embodiment.

As an option, the block diagram of general process 74 of FIG. 5 may beviewed in the context of the details of the previous Figures. Of course,however, the block diagram of general process 74 of FIG. 5 may be viewedin the context of any desired environment. Further, the aforementioneddefinitions may equally apply to the description below.

It is appreciated that general process 74 may include embodiments ofeither or both of surveying software 21 and server software 23 forperforming the following actions. It is appreciated that the actions ofgeneral process 74 can be mapped into the actions of surveying software21 and/or server software 23, or vice versa.

General process 74 may start with action 75 by identifying an objectpresented on the screen display of a computing equipment such as object10 displayed on screen display 16 of computational device 11 of FIG. 1 .For example, action 75 may identify objects by comparing with objects ofobjects database 72.

Action 76 may obtain (e.g., by a screenshot) the complete image(picture) of the object from the screen display, once completeness isvalidated. For example, the completeness can be validated by determiningthat sufficient white space (or non-image space) is present on all sidesof the picture, which determines that the image is shown in its fullform.

Once completeness is validated, action 77 may crop the relevant imagefrom the screenshot taken based on a selected machine vision algorithm.

Action 78 may identify the object of the cropped image by machine visionand implementing (associating) a unique digital signature for the objectof the cropped image. Action 78 may update and maintain the occurrences(threads) database 73.

Action 79 may search for, and identify a unique signature. The digitalsignature may be implemented as a process of analyzing the image digitalbinary, and searching for three-pixel-spots binaries, which comprise theimage key. Such digital signature may be used as a watermark for binaryfiles.

If the unique signature is within the applicable signatures repositoryof objects (action 80) that are of interest (e.g., as provided by a list41), action 81 may operate to process the cropped image to extract otherrelevant text and URL data presented within the cropped image, or in thevicinity of the cropped image, or elsewhere on the same page asdisplayed with the cropped screenshot. Action 81 may also update objectsdatabase 72

For example, action 81 may scan the cropped image, or its vicinity, toextract relevant text, such as a brand name. Action 81 may use, forexample, optical character recognition (OCR) to determine a caption, alogo, a brand name, etc., within the cropped image or its surrounding.

Action 82 may optionally analyze and determine, based on the extractedobject image, and object parameters, including text, and URL data, ascaptured from the screen display, the occurrence of a particularadvertisement, being displayed to the user on the screen display.

Action 83 may optionally associate the particular advertisement with anyof the other objects identified to be displayed to the user on thescreen display. Such associations of advertisements with objects (e.g.,of objects database 72) and with threads (e.g., of occurrences database73) may be maintained in an advertising database 84. In this respect,the general process 74 of FIG. 5 may be divided into the two (or more)processes. For example, general process 74 may be divided into theprocess of surveying software 21, typically executed by computationaldevice 85 operated by user 13, as shown and described with reference toFIG. 3 , and the process of server software 23, typically executed by(one or more) network server 15, as shown and described with referenceto FIG. 4 .

The purpose of the division general process 74 is to reduce theprocessing load on the processor of the computational device 86, and toreduce the communication load on the network connecting thecomputational device 87 and network server 15, by selecting, inreal-time, the parts, or elements, or components, of the contentdisplayed to user 13 on the display of computational device 88. Suchselection is performed automatically based on object characterizationdata, such as list 41, provided in run-time by network server 15 tocomputational device 89.

Therefore, computational device 90 may screenshot the screen selectivelyand communicate the screenshot data to the network server 15. And thenetwork server 15 may then crop the image, for example using imagerecognition. Network server 15 may then deduce image characterizationand/or metadata and decide, in real time, if it is relevant. Networkserver 15 may then communicate to computational device 91 requests toretrieve (e.g., by screenshots) and communicate further elements of thedisplayed content.

The image analysis of the general process 74 of FIG. 5 is thereforedivided, in real-time, between computational device 92 and networkserver 15, so that lightweight parts (non-processor intensive) of theanalysis are performed by computational device 93, thus reducingcommunication load, and heavyweight parts (processor intensive) of theanalysis are performed by network server 15, thus reducing processingload on computational device 94.

Lightweight parts of the analysis may include analyzing clickpath data,HTML content, available image metadata, image code, description text incaption, iframe and JavaScript wrappers, URL text and URL parameters,etc. Heavyweight parts of the analysis may include image recognition,OCR, identifying image elements such as logo, brand name, etc., digitalsignature analysis, etc., as well as identifying any particularadvertisement displayed to user 13.

Thereafter, network server 15 may associate, and/or correlate, images inthe stream of data or content received by computational device 95 as itis actually displayed to user 13, as well as associate, and/orcorrelate, displayed images and displayed advertisements. Suchassociations, and/or correlations, may then be used to improve theselections, and/or orientations, of display screenshots, image cropping,and image analysis.

The term orientation may refer to the location and timing of thescreenshot, operating multiple screenshots to retrieve a complete image,division of the analysis, etc.

Advertisement analysis and association and/or correlation may includedetermining the advertiser, the brand name, the advertisement landingpage (URL), etc. Some of the advertisement analysis may be executed bythe network server 15, for example using image processing, and some ofthe advertisement analysis may be executed by the computational device96, for example by analyzing clickpath data, image metadata, etc.

It is expected that during the life of this patent many relevantadvertisement systems and applications will be developed and the scopeof the terms herein, particularly of the terms “mobile device”, “mobileapplication”, “advertisement” or “mobile advertisement”, “advertisementbidding”, and “bidding agent” or “ad-network”, is intended to includeall such new technologies a priori.

Additional objects, advantages, and novel features of the embodimentsdescribed herein will become apparent to one ordinarily skilled in theart upon examination of the following examples, which are not intendedto be limiting. Additionally, each of the various embodiments andaspects of embodiments as delineated hereinabove and as claimed in theclaims section below finds experimental support in the followingexamples.

It is appreciated that certain features of the system and methoddescribed herein, which are, for clarity, described in the context ofseparate embodiments, may also be provided in combination in a singleembodiment. Conversely, various features, which are, for brevity,described in the context of a single embodiment, may also be providedseparately or in any suitable sub-combination.

Although the system and methods has been described in conjunction withspecific embodiments thereof, it is evident that many alternatives,modifications and variations will be apparent to those skilled in theart. Accordingly, it is intended to embrace all such alternatives,modifications and variations that fall within the spirit and broad scopeof the appended claims. All publications, patents and patentapplications mentioned in this specification are herein incorporated intheir entirety by reference into the specification, to the same extentas if each individual publication, patent or patent application wasspecifically and individually indicated to be incorporated herein byreference. In addition, citation or identification of any reference inthis application shall not be construed as an admission that suchreference is available as prior art.

What is claimed is:
 1. A method for monitoring graphical content of ascreen display, the method comprising: receiving, by a software programexecuted by a processor of a computerized device communicatively coupledto a communication network, at least one parameter characterizing agraphical object; monitoring a display of the computerized device,wherein the processor of the computerized device receives at least oneadvertisement via the communication network, wherein the processordisplays the at least one advertisement on a display screen of thecomputerized device, wherein the monitoring uses the software programexecuted by the processor of the computerized device; recognizing, bythe software program executed by the processor of the computerizeddevice, the at least one graphical object being displayed on the displayscreen; and collecting: the graphical object as displayed on the displayscreen; and at least one of: a parameter associated with the displayscreen; and a parameter associated with other content displayed on thedisplay screen substantially concurrently.
 2. The method according toclaim 1, additionally comprising: identifying the other contentassociated with the parameter associated with the collected graphicalobject as an advertisement.
 3. The method according to claim 1,additionally comprising: communicating to a server via the communicationnetwork at least one of: the recognized graphical object; and the atleast one parameter.
 4. The method according to claim 1, additionallycomprising: the at least one parameter being at least one of: temporallyassociated with the recognized graphical object; spatially associatedwith the recognized graphical object; and a mode of display associatedwith the recognized graphical object.
 5. The method according to claim1, additionally comprising: determining the least one graphical objectto recognize to be at least one of motion graphics and motion image;determining at least one of number of samples, rate of sampling, andperiod of sampling for recognizing the graphical object; and repeatingthe recognizing the graphical object according to the at least one ofnumber of samples, rate of sampling, and period of sampling forcapturing the graphical object.
 6. The method according to claim 1,wherein the recognizable graphical object comprises at least two partsof the recognizable graphical object, wherein each of the at least twoparts of the recognizable graphical object is associated with at leastone parameter associated with at least one of: the display screen; andother content displayed on the display screen substantiallyconcurrently.
 7. A computer program product embodied on a non-transitorycomputer readable medium comprising computer code for: receiving, by asoftware program executed by a processor of a computerized devicecommunicatively coupled to a communication network, at least oneparameter characterizing a graphical object; monitoring a display of thecomputerized device, wherein the processor of the computerized devicereceives at least one advertisement via the communication network,wherein the processor displays the at least one advertisement on adisplay screen of the computerized device, wherein the monitoring usesthe software program executed by the processor of the computerizeddevice; recognizing, by software program executed by the processor ofthe computerized device, the at least one graphical object beingdisplayed on the display screen; and collecting: the graphical object asdisplayed on the display screen; and at least one parameter associatedwith at least one of: the display screen; and other content displayed onthe display screen substantially concurrently.
 8. The computer programproduct according to claim 7, additionally comprising computer code for:identifying the other content associated with the parameter associatedwith the collected graphical object as an advertisement.
 9. The computerprogram product according to claim 7, additionally comprising computercode for: communicating the recognized graphical object to a server viathe communication network.
 10. The computer program product according toclaim 7, additionally comprising computer code for: collectingparameters associated with the recognized graphical object theparameters including at least one of: temporal parameters associatedwith the capturing of the recognized graphical object; spatialparameters associated with the recognized graphical object; and a modeof display associated with the recognized graphical object.
 11. Thecomputer program product according to claim 7, additionally comprisingcomputer code for: determining the least one graphical object to captureto be at least one of motion graphics and motion image; determining atleast one of number of samples, rate of sampling, and period of samplingfor capturing the graphical object; and repeating the recognizing thegraphical object according to the at least one of number of samples,rate of sampling, and period of sampling for capturing the graphicalobject.
 12. The computer program product according to claim 7, whereinthe recognizable graphical object comprises at least two parts of therecognizable graphical object, wherein each of the at least two parts ofthe recognizable graphical object is associated with at least oneparameter associated with at least one of: the display screen; and othercontent displayed on the display screen substantially concurrently.